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Career counselors continually evaluate their work; they draw conclusions and develop 
action plans based on numerous client activities: homework completion, client 
engagement, acquisition of interview skills or relaxation skills, number of employers 
contacted, and so forth. Both counselors and clients typically know when counseling is 
successful. Unfortunately, the evidence used to gauge success often is not considered 
evaluation, is not documented, and therefore cannot be used to back up claims that 
counseling has been successful. 

Conger, Hiebert, and Hong-Farreil (1993) found that counseling is rarely evaluated in 
Canada. In some sectors, 40% of counselors reported never formally evaluating their 
work. This is unfortunate, since research has shown that career guidance and 
counseling consistently produce positive results. (See Killeen & Kidd, 1991 ; Oliver & 
Spokane, 1988.) In fact, Killeen and Kidd found that 90% of the studies recorded some 
positive counseling effects. Given these findings, it is surprising that counselors are not 
eager to evaluate such a predictably beneficial experience. Perhaps a new approach to 
evaluation is needed; one that counselors see as relevant, practical, and capable of 
embracing the informal observations that counselors and clients make about counseling 
progress. 

AN ALTERNATIVE FRAMEWORK 



Counseling is interactive. Client-need determines counseling intervention. The 
intervention is tailored to client characteristics and desired outcome. As counseling 
progresses, counselors adjust the approach to fit the client's changing situation. 
Evaluation models must accommodate this interplay between counselor and client and 
between process and outcome. 

Figure 1 depicts a framework connecting long-term global impacts of counseling (e.g., 
job satisfaction, employability, career maturity); immediate outcomes associated with 
counseling (changes in client attitudes, knowledge, skills); client engagement in the 
counseling process; and, counselor approaches. 

The loops in the framework illustrate the interactive nature of counseling. 

Process-Outcome Loop. 

Process and outcome interact in a circular fashion. Certain processes foster particular 
types of learning and in turn, create certain types of global impact. Reciprocally, the 
types of client skill, knowledge, and attitude necessary to achieve a certain kind of 
global impact can be identified, and the process needed to facilitate that learning can 
also be identified. 



FIGURE 1 
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1. Evaluation Components 
(Connections: 2,3,4) 

ft 

2. Counseling Process 
(Connections: 1 ,3,5,6) 



3. Counseling Outcome 
(Connections: 1,2, 7, 8) 

ft 

4. System Requirements 
(Connections: 1 



5. What is the counselor doing: 
Skills, Strategies, Tools. 
(Connections: 2, 5, 6, 7, 8) 



6. What is the client doing? 

Skill Practice, Homework, Self-Examination. 
(Connections: 2,5,7) 
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7. What did the client learn? 

Skills, Knowledge, Attitudes. 
(Connections: 3, 5, 6, 8) 

ft 

8. What difference did the learning 
make? Precursors, Presenting problem, 
Socio-economic impact. 

(Connections: 3,5,7) 



Process Loop. 

Evaluating the counseling process requires detailed information which links counselor 
activities and client reactions. This helps identify the processes that promote client 
change and that aid the development of alternative plans when sufficient progress is 
lacking. Client documentation might include engagement in the counseling process, 
homework completion, client openness and honesty, and client follow-through. Such 
data demonstrate that clients are doing their part in counseling. Evidence depicting the 
counselor as an indispensable part of the process might include the pattern of 
micro-skills used, the focus in a counseling session (both content and process), and 
data showing that an acceptable procedure for an intervention followed. These factors 
can be assessed from counselor case notes, client check-lists, semi-structured 
interviews, or formal questionnaires. 

Outcome Loop. 

Counselors need to be clear about the legitimate outcomes (effects or products) of 
counseling (Hiebert 1 989, 1 994; Killeen & Kidd, 1 991 ; Killeen White, & Watts, 1 993). In 
Figure 1 , counseling outcomes have two major components: 

(1) Learning outcomes: the knowledge, skills, and attitudes that are directly linked to 
counseling. These are the legitimate outcomes of counseling: 



Page 4 of 8 



ED414519 1995-00-00 A Changing Focus in Evaluation: Linking Process and Outline: 

ERIC Digest. 



ERIC Resource Center 



www . eric . ed . gov 



knowledge about self, the labor market, job descriptions, entrance qualifications, and 
how to overcome barriers 



skills for decision making, job interviews, self-management, making transition, and 
overcoming barriers like anxiety, substance abuse, poor financial planning 



attitudes towards being planful, belief in self, motivation to look for work, self-esteem, 
increased optimism Learning outcomes arise both from counseling and from learning to 
apply existing skills to new contexts (e.g., using communication skills in conflict 
resolution). These are the legitimate outcomes of counseling, outcomes that counseling 
can reasonably hope to influence. 

(2) Global outcomes reflect counseling's larger impact on the client's life. They include: 



changes in client-presenting problem 



getting along better with co-workers 



increased job satisfaction 



less stress during job interviews 



fewer tension headaches 



socio-economic outcomes 
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job retention 



length of time to job offer 

Administrators and sponsors often focus on global impact outcomes, but outcomes are 
influenced greatly by factors over which counseling has little control (e.g., the number of 
jobs available, corporate climate, etc.). Ultimately, it is important to demonstrate that 
counseling has affected these variables, but it is also important to refrain from promising 
that counseling can facilitate change in areas over which it has little control. 

System Requirements. 

In addition to the above factors, program administrators and sponsors often are 
interested in factors such as adherence to mandates, types of clients, types of client 
problems, and client satisfaction with service. These variables are important to the 
functioning of the agency, but they are not indicators of the effectiveness of counseling 
per se. To emphasize this difference, they are called "system requirements." System 
requirements are part of the policies and procedures governing an agency and 
departures from these requirements should be negotiated between the counselor and 
the agency manager or sponsor. 

INFORMAL MEASURES 



Although standardized assessment is emphasized in counseling evaluation, there is 
frequent incongruence between the aims of standardized tests and the needs of clients 
(Killeen & Kidd, 1 991 ). Implicit in Figure 1 is a far greater emphasis on informal 
procedures that document the judgments counselors and clients make about counseling 
progress. These include checklists for homework completion, skill mastery, skill 
implementation, steps completed in a program; subjective ratings of affective state (e.g., 
depression, motivation, stress level, job satisfaction); use of job interview skills during 
job interviews (see Hiebert (1991, 1994a, 1994b) for specific examples). 

Currently, these informal measures are in their infancy. Better ways must be developed 
to track variables that are part of the counseling process and which have an influence 
on client change. For example, "planful" attitude is an important prerequisite (or 
co-requisite) to developing a career-action plan. Therefore, it is important to have a 
trustworthy and easy-to-use procedure to track changes in such attitudes. 

SUMMARY: A CALL FOR ACTION 



Two points underline the main arguments in this paper. 
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1. Counselors, program administrators, and sponsors need to reformulate their view of 
evaluation so that it provides an essential link between process and outcome; and, is an 
integral part of counseling, coequal with relationship building and intervention planning. 

Evaluation needs to be planned and implemented alongside client-change 
intervention-not conducted at the end of a program by an external expert. 



2. The scope of evaluation (what constitutes acceptable evidence) needs to be 
expanded to include the sorts of data that counselors and clients already collect on a 
regular basis. This includes client self-monitoring data, homework data, quantification of 
counselor observations in case notes, documentation of client in-session skill practice, 
goal attainment scaling, and performance assessments. 

An evaluation model should assess the informal observations counselors and clients 
use to indicate whether they are on the right track, the amount of progress they are 
making, and the achieving of desired outcomes. It should encourage counselors to 
develop creative ways for documenting and quantifying those observations, and it 
should create non-quantified ways of portraying the evidence that clients, managers and 
supervisors find acceptable. This will ensure that evaluation needs are seen as relevant 
by ail concerned: clients, counselors, agency managers, district supervisors, and 
funders. 
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